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Introduction 


In 1986, the National Research Council (NRC) of the United States recom- 
mended that the country should pay attention to STEM (Science, Technology, 
Engineering, Mathematics) education. In 2006, the American Competitiveness 
Initiative (ACI) program pointed out that the key to global competitiveness 
is talent with STEM literacy. The Next Generation Science Standards (NGSS) 
announced by the US NRC in 2013 officially listed STEM education in the cur- 
riculum standards. The Chinese Academy of Educational Sciences published 
the White Paper on STEM Education in China in 2017 and launched the China 
STEM Education 2029 Innovation Action Plan. In 2016, the American Rhode 
Island School of Design (RSD) launched the“STEM to STEAM (Science, Technol- 
ogy, Engineering, Art, Mathematics)” campaign, adding Art to the old STEM 
education, in order to enhance innovation and creativity. The “Great Future of 
Work” report of the 2016 World Economic Forum mentioned that STEAM can 
train children as skillful people and has become a key goal of global education 
reform. It is a cross-disciplinary teaching framework for developing students’ 
ability to learn hands-on tinkering, real-world problem solving, practicing 
hardware and software skills, and conducting scientific inquiry. 

Since 2018, Taiwan has begun to extend from providing nine years of ba- 
sic education to twelve years. With a stronger emphasis on STEAM education, 
the new curriculum is expected to foster the next generation in transforming 
subject knowledge into skills in solving real-world problems. The syllabus of 
the natural sciences in the curriculum incorporates building scientific models 
and the use of models for scientific inquiry. According to different learning 
stages, the process of using a model to express the inquiry for solving scien- 
tific problems through self-reflection and peer discussion can be cultivated. 
Many scholars have found that students have limited views on models and 
modeling. While teachers often view models as static knowledge and focus 
on the content of specific models, students often do not discuss the ideal, 
temporary and pluralistic models in detail (Justi & Gilbert, 2002). The research 
of Pata and Sarapuu (2006) observed that when designing teaching activities 
and improving students’ modeling ability, science teachers can be divided into 
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Abstract. This research aimed to 

explore the effects of a virtual lab (VL) 

and a Microcomputer-based Lab (MBL) 

on students’ performance in scientific 
modeling. A web-based virtual lab anda 
low-cost MBL were proposed to help first- 
year engineering students build scientific 
models. Empirical research was done ina 
slope motion experiment. The participants 
were 178 first-year engineering students 

in Taiwan, and they were divided into a 

VL group and an MBL group. From the 
results of the questionnaire, these groups 
of students thought that the systems were 
usable and easy to use, and they expressed 
positive attitudes towards the labs. The 
post-test’s average score was higher than 
that of the pre-test for both groups, and 
the average posttest score of the VL group 
was better than that of the MBL group. 
From the students’ learning sheets, many 
students successfully manipulated the 
experimental variables and built correct 
models after gradual revision of earlier 
models. According to the results of the 
empirical research, these systems helped 
the students understand the meaning of 
the experiment and increased students’ 
interests with hands-on labs. A comparison 
of the results of these two groups\suggests 
the integration of VL and MBL to facilitate 
students’ learning. 
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two orientations: those who focused on using the model as a tool for expression and those who emphasized the 
model as an exploration tool (Pata & Sarapuu, 2006). For the former group, the teaching goal focuses on develop- 
ing students’ ability to model and use models to describe a systematic scientific phenomenon, e.g., Hanke (2008). 
For the latter group, the teaching method focuses on guiding students to use the model to reason in the process of 
scientific inquiry, and to compare and correct their own mental models, e.g., Justi and Gilbert (2002) and Schwarz 
and White (2005). 

Computer simulation is a powerful tool for scientific discovery learning (SDL) since it can provide an explor- 
atory learning environment (Reid, Zhang and Chen, 2003). There are many empirical studies on students’ learning 
of scientific modeling with computer simulation (e.g., Cheng et al., 2017; Finkelstein et al., 2005; Gregorcic & Bodin, 
2017; Rutten, van der Veen, & van Joolingen, 2015; Wang, Guo, & Jou, 2015). These studies put emphasis on cultivat- 
ing students’ scientific research skills in building scientific models. Some apply virtual lab to education, e.g., Wang, 
Chang, Hwang, and Chen (2018) developed an educational game based on microworld and encouraged students 
to use the mathematics acquired in classroom to explore, discover and solve practical problems. The test results 
show that virtual learning methods are better than traditional ones. Rojano and Garcia-Campos (2017) pointed out 
that the feedback from intelligent support systems and virtual lab at critical moments in modeling activities, such 
as how to understand the target phenomena, build models and do predictions, were critical to their building of 
spreadsheet models. Gobert, Kim, Sao Pedro, Kennedy and Betts (2015) developed a virtual lab system for life science 
courses that helped students learn about the ecological environment and food web of aquatic organisms. In SDL 
activities, if interpretative support (IS), experimental support (ES) and reflective support (RS) are included to help 
the learner complete the experiment, it will be more effective and can improve the learner's reasoning ability (Reid, 
Zhang and Chen, 2003). The above studies have shown that virtual laboratories provide simplified real-world models 
to make it easier for learners to understand scientific phenomena. Such labs also allow learners to conduct experi- 
ments in a faster and easier manner, and provide immediate feedback when students make mistakes, giving them 
the opportunity to immediately repeat the same experiments by avoiding the same mistakes (Heradio et al., 2016). 

Even though simulations of physics labs are common practice, some teachers still believe that physical labs 
provide good training for students. At present, the physical labs on campus can be divided into non-computerized 
laboratories and computerized laboratories. Most computerized labs allow students to use Microprocessor-based 
Lab (MBL) systems to run physical experiments. An MBL system can collect large amounts of data in a short time 
and plot the data into charts (Barclay, 1985; Bermudez-Ortega, Besada-Portas, L6pez-Orozco, Bonache-Seco, & De la 
Cruz, 2015; Brasell, 1987; Deniz & Dulger, 2012; Liu, Wu, Wong, Lien, & Chao, 2017; Mokros, 1986; Nicolaou, Nicolaidou, 
Zacharia, & Constantinou, 2007; Tortosa, 2012; ). Most of the MBL systems found on campus are commercially avail- 
able MBL systems, such as those from LEGO MINDSTORMS (Church, Ford, Perova, & Rogers, 2010), Fathom Dynamic 
Statistics (Erickson, 2006), PASCO Capstone and Vernier Asia. However, due to the high cost of commercial tools, 
some researchers have developed their own experimental kits at much lower cost (e.g., Bermudez-Ortega et al., 
2015; Chen et al., 2012; Kuhn & Vogt, 2013; Liu et al., 2017). The results showed that compared to non-computerized 
laboratories, regardless of whether it was a commercial system or not, MBL helped students’ learning. 

This research proposes both a virtual lab system based on Web technology and a low-cost MBL system, aiming 
to explore and compare the benefits of the two systems for students in running physics labs with a modern data- 
logger and a web-based simulation tool and in doing scientific modeling. After a comparison, an integration of both 
approaches can be proposed to take advantage of the benefits of each approach. In order to evaluate the usability of 
the two systems, an empirical research was conducted on slope motion experiments in a first-year university physics 
lab course, and 118 students were assigned into an MBL group and a virtual lab (VL) group. The research focused 
on using a model as a tool for inquiry. According to the Model-Enhanced Thinker Tool Course (METT) advocated 
by Schwarz and White (2005), the development of computer simulation modeling courses is still in the exploratory 
Stage. In order to examine the benefits of doing modeling, more empirical studies are needed. A pretest, a posttest 
and a learning sheet were used for examining the learning outcomes of the students. To assess the usability of the 
systems, the authors revised the Digital Platform Usability Questionnaire by Chou and Lu (2014). The questionnaire 
adopted Likert’s five-point scale and was divided into four parts with four subscales: perceptual usefulness, perceived 
ease of use, use attitude and willingness to use. The remainder of this paper is organized as follows. Methodology 
of Research will introduce the implementation of the empirical research, the architectures of the virtual lab system 
and the MBL system. The Results of Research section presents the results of the experimental learning sheet and 
the students’ responses to the questionnaire. The Discussion section discusses the issues of the proposed systems 
based on the empirical results. Finally, the Conclusions section presents the contributions of this research. 
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Research Problem 


As suggested by the above literature, virtual labs and MBLs could help students to explore and build scien- 
tific models, but there were still some shortcomings with both approaches. Researchers believed that virtual and 
hands-on labs were not the only alternatives, but that they could be used in combination to enhance student 
learning (Abdulwahed & Nagy, 2011; Simonova, 2010; Wiesner & Lan, 2004). This research proposed a virtual lab 
system based on Web technology and a low-cost MBL system to help students build scientific models, explored and 
compared the effectiveness of using two systems for scientific modeling with an empirical research and proposed 
how to integrate both approaches as suggested by the students’ feedback . The research questions were as follows: 

1. How well does the web-based virtual lab system help students do experiments and build scientific 
models? 
2. How well does the MBL system help students do experiments and build scientific models? 


Research Focus 


Harrison and Treagust (1996) pointed out that it is easy for learners to understand abstract scientific concepts 
through models. Harrison and Treagust (2000) also believed that models can facilitate inquiry, understanding and 
communication, and served as the primary tool for scientific thinking and activities. Treagust, Chittleborough and 
Mamiala (2004) pointed out that if teachers only provide static descriptive models and do not emphasize the ma- 
nipulation and prediction of models, it would be difficult for students to learn modeling. Therefore, researchers have 
tried to improve students’ modeling ability and enhance scientific concept learning through traditional hands-on 
labs (Gomes & Bogosyan, 2009), virtual lab (Finkelstein et al., 2005; Gobert et al., 2015; Gobert et al., 2011; Grego- 
rcic & Bodin, 2017; Rojano & Garcia-Campos, 2017; Rutten, van der Veen, & van Joolingen, 2015; Wang et al. , 2018; 
Wang, Guo & Jou, 2015) and MBL (Bermudez-Ortega et al., 2015; Church et al., 2010; Erickson, 2006; Liu et al., 2017). 

Traditional hands-on labs allow students to actually experiment with the advantages of haptic skills and in- 
strumentation awareness (Abdulwahed & Nagy, 2011), but such labs also encounter the problems of data logging, 
equipment, space and maintenance staff costs (Gomes & Bogosyan, 2009). Virtual labs reduce experiment costs 
and have simplified real-world models to make it easier and safer for learners to experience the phenomena with 
computer simulation. Allowing users to conduct experiments with a user-friendly graphical interface, virtual labs 
can immediately provide feedback on the mistakes made by the students, giving them the opportunity to imme- 
diately repeat the same experiments and high-cost experiments. Although virtual labs solved some shortcomings 
of the hands-on labs, it did not allow students to acquire haptic skills and instrumentation awareness. Therefore, 
some researchers believe that each type of lab has its own advantages, so the challenge is how to combine the 
two laboratories in a complementary way to achieve specific learning effects (Abdulwahed & Nagy, 2011; Heradio 
et al. , 2016; Wiesner & Lan, 2004; Zacharia, 2007). 

Researchers and practitioners continue to develop tools to assist students in scientific modeling, but it is also 
an important issue to improve the teaching effectiveness of modeling by arranging sequences of modeling teach- 
ing. Schwarz and White (2005) proposed the Model-Enhanced Thinker Tools curriculum (METT), which pointed out 
that using the scientific inquiry process as the main axis, the application of computer software helps students build 
and refine the proposed model. Schwarz and Gwekwerere (2007) proposed the curriculum design framework of 
Engage-Investigate-Model-Apply (EIMA) and considered a streamlined curriculum design and the flexible space 
of the lesson plan. According to the modeling teaching sequence advocated by the above researchers, the devel- 
opment of computer simulation modeling course is still in the exploration stage. For the improvement effect of 
modeling teaching, more empirical research is still needed. 

This research aimed to propose a virtual lab with Web technology and a low-cost MBL system based on an 
Arduino development board, in order to explore the effectiveness of the two systems for helping students in sci- 
entific modeling. The results could inspire on how to combine MBL and Virtual lab in the future to take advantages 
of both approaches. In order to evaluate system feasibility, an empirical research was conducted in the university’s 
first-year physics experiment course with the METT framework of teaching modeling (Schwarz & White, 2005). 
The virtual lab based on Web technology allows students to simulate slope motion on Apple computers, PCs, or 
mobile devices. This system offers a learning environment for exploring physical phenomena and lets students 
adjust experimental variable parameters, show an animation of the motion, and then generate experimental data. 
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The MBL system allows students to run hands-on physical experiments and has the advantage of collecting large 
amounts of data in a short period of time. Instead of using a commercially available MBL system, they use a low- 
cost data logger developed by the authors of this paper. 


Research Methodology 


General Background 


This research developed a VL system and an MBL system and conducted an empirical research in the first 
year of physics experiment courses at a university in Taiwan in February 2019. In order to explore how the system 
benefitted the students, MEET’s framework of teaching modeling was implemented to conduct an empirical re- 
search (Schwarz & White, 2005). A post-test questionnaire, which was modified from the Digital Platform Usability 
Questionnaire proposed by Schwarz and White (2005), was adopted to evaluate the benefits of the two systems. 


Sample 


In order to evaluate the benefits of the virtual lab and the MBL system, an empirical research was conducted 
on the slope motion experiment in the first-year physics lab course of a university in Taiwan in February 2019. One 
of the authors was an instructor of the physics lab course for two classes of students in two engineering depart- 
ments with similar scores of academic competence. The university administration encouraged this type of action 
research with the purpose of helping instructors develop practical solutions to address learning problems. One 
class with 60 students was randomly assigned to the Virtual Lab (VL) group and the other class with 58 students 
was assigned to the MBL group, with a total of 118 students. 


Instrument and Procedures 


A. Instrument 
1) Virtual lab 

The virtual lab architecture developed in this research is shown in Figure 1. The virtual lab is presented as 
a website, and users can run experiments without installing software packages. In the client side of Figure 1, us- 
ers use mobile devices for modeling activities such as PCs, laptops, tablets or smartphones. The server is mainly 
responsible for providing the functions of the virtual lab, webpage rendering and processing the messages sent 
back by the user. For the front end, the web application is mainly implemented with Vue.js, where CSS is used to 
improve the graphical user interface. For the back end, an Apache server with PHP's Laravel architecture provides 
the routing and mediation mechanism of the website. Laravel’s Object Relational Mapping (ORM) technology is 
employed to connect with the database. 


Figure 1 
Virtual lab architecture 
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The workflow of the virtual lab is shown in Figure 2. The first phase introduces the purpose, principles, and 
experimental instruments of the experiment, giving each student a basic understanding of the experiment. In 
the second phase, the student sets up a virtual lab environment with virtual components (Figure 3). In the third 
phase, the student chooses the independent and dependent variables of the experiment for data collection and 
observing the simulation of the experiment. In this research, the user interface is designed to facilitate the students 


in building scientific models (Figure 4). 


Figure 2 
The workflow of virtual lab 
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2) MBL 


The MBL system architecture is shown in Figure 5. Students must first set up the lab equipment including 
the data logger at the designated location. When the experiment runs, the data logger automatically captures the 
experimental data and saves them to a computer as a file. Finally, the students open the experimental data file, and 
draw a chart with the data. The workflow of the lab is shown in Figure 6. During the experiment, students are free to 
manipulate the independent variables and analyze the charts of the experimental data and build scientific models. 

The data logger of the MBL system is built upon the Arduino UNO expansion board and the sensor of each 
experiment. For example, for the slope movement or free fall experiment, the distance of the moving object is 
measured with the ultrasonic sensor HC-SRO4. In the RC charge and discharge experiment, the KSM048 voltage 
sensing module measures the voltage during the charging and discharging of the capacitor. In the pendulum 
experiment, the MPU6050 sensing module measures the angle of the pendulum. 

When an experiment is run, the experimental data is transmitted to the computer and saved as a CSV file. To 
make it easier to observe experimental data, students can use Microsoft Excel software to open this CSV file and 
plot the data for analysis. 


Figure 5 
MBL system architecture 
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B. Procedures 
This research was based on the modeling teaching framework proposed by Schwarz and White (2005). The 


METT course proposed by Schwarz et al. included the following steps: (1) problem, (2) hypothesis, (3) investiga- 
tion, (4) analysis, (5) model, (6) evaluation. In order to assess the effectiveness of the course, before the start of the 
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experiment, a pretest related to the slope experiment was conducted for the two groups of students. During the 
experiment, the research provided a work sheet to assist the two groups of students to record the experimental 
process. After the experiment, the students were given a posttest. Finally, the students were asked to fill out a 
course questionnaire. The design activities of this research are briefly described below. 


1) Virtual Lab group 


Table 1 shows the empirical research curriculum for the VL group. The activity consists of the following steps: 

1. Pre-test: Check the knowledge level of students on slope movements. 

2. Prediction: Use the application questions to propose different variables (such as mass and slope angle) 
and let the students predict the experimental results. 

3. Experimentation: Students use the virtual lab to simulate experiments according to the work sheet. 
By selecting the experimental variables, they observe the physical characteristics and record the ex- 
perimental results. 

4. Interpretation: Explain how the mass of the object and the slope angle affect the acceleration of the 
object based on the experimental data. 

5. Model application: Students answer the application questions of the prediction step and explain the 
reasons. 

6. Evaluation: The application question of the final model confirms whether the experimental results are 
consistent with the final model. 

7. Post-test: Evaluate whether the student’s understanding of the slope experiment is improved during 
this modeling process. 

8. Questionnaire: Evaluate the virtual lab. 


Table 1 
Virtual group empirical research course planning form 


Time Course contents 
0.5 hour Pre-test 
0.5 hour Predictive activities 
0.5 hour The instructor introduces the lab and how to run the virtual lab. 


Virtual lab learning content: 
1. Students learn the physical phenomena of the experiment. 


nel 2. Students set up a virtual lab environment in the system. 
3. Run experiment and record data. 
2 hours Interpretation, Model application and Evaluation 
0.5 hour Post-test 
0.5 hour Students fill out the questionnaire. 


2) MBL group 


Table 2 is the MBL group empirical research curriculum planning table. The activity consists of the following 
steps: 
1. Pre-test: Check the knowledge level of students on slope movements. 
2. Prediction: Use the application questions to propose different variables (such as mass and slope angle) 
and let the students predict the experimental results. 
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3. Experimentation: Students use the MBL system to run experiments according to the worksheet. By 
selecting the experimental variables, they observe the physical characteristics and record the experi- 
mental results. 

4. Interpretation: Explain the relationship between the acceleration of the sliding object, its mass and the 
slope angle based on the experimental data. 

5. Model application: Students answer the application questions of the prediction step and explain the 
reasons. 

6. Evaluation: The application question of the final model confirms whether the experimental results are 
consistent with the final model. 

7. Post-test: Evaluate whether the student's understanding of the slope experiment is improved during 
this modeling process. 

8. Questionnaire: Evaluate the virtual lab. 


Table 2 
MBL group empirical research course planning form 


Time Course contents 
0.5hour Pretest 
0.5hour Prediction 
0.5 hour The instructor introduces the lab and how to run the lab. 


Experimental observation: 
1. Setting up an experimental environment. 


me 2. Using the MBL system to collect and observe the experimental data. 
3. Using a stopwatch to measure and observe the experimental data. 
2 hours Interpretation, Model application and Evaluation 
0.5 hour Posttest 
0.5 hour Students fill out the questionnaire. 


Data Analysis 


The questionnaire for this research was modified from the Digital Platform Usability Questionnaire presented 
by Chou and Lu (2014). Based on the Technology Acceptance Model (TAM) proposed by Davis (1989), the research 
was developed with four subscales: perceptual usefulness, perceived ease of use, attitude of use, and willingness 
to use. Perceptual usefulness is the degree to which the user feels that the system is easy to use. Perceived ease 
of use is the degree to which the user believes that the system can be used to actually learn specific knowledge. 
Both of these aspects will affect the attitude of use, further affecting the willingness to use. The questionnaire was 
measured using a 5-point Likert scale, ranging from 1 (disagree very much) to 5 (agree very much). The higher the 
score, the more the learner agrees that the digital learning system is usable. The questionnaire was evaluated with 
195 users of the Digital Learning Center in Taiwan (aged 20-30 years old), and each subscale’s Cronbach a value 
was above .80. The paired samples t-test and correctness rate were used to analyze the scores of the pre-test and 
the post-test. 


Research Results 
A. Work sheet 


The work sheet is mainly used to record the experimental data by the students. During the experiment, the 
students explored the influence of the mass of the sliding object and the slope angle on the experimental results, 
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and then built a scientific model. Figure 7 (a) and (b) are the work sheets of a student in the VL group. They experi- 
mented with the distance of the object sliding over time when the angle was fixed at 15 degrees and the mass of 
the object was 1000g and 2000g. And the distance the object moved with time when the mass of the object was 
1000g and the slope angle was 25 degrees and 10 degrees respectively. It can be known from Figure 7 (a) that the 
travelled distances of the objects of different masses over time are similar. It can be seen from the figure (b) that 
the slope angles were different, and the objects of the same mass travelled different distances per unit time. It is 
further known that when the angle was larger, the travelled distance of the object in unit time increased, indicating 
that the object was sliding with increasing speed. 

Figure 7 (c), (d) and (f) are the work sheets of one of the students in the MBL group. As shown in Figure 7 (c), 
there are two objects of masses 270.98g and 372.25g, and the slope angle is fixed at 5 degrees. In Figure 7 (d), the 
object mass is 372.25g, and the slope angles are 5 degrees and 10 degrees respectively. Figure 7 (f) compares the 
results of the experiment when the students used MBL and stopwatch. From Figure (c), it can be found that the 
slope angle is the same, but the travelled distances of the two objects of difference masses were slightly different. 
Some students thought that this was an experimental error caused by friction or an operational error, indicating 
that the object mass did not affect the sliding speed of the object. However, some students believed that the mass 
of objects affected its speed in slope motion. From Figure (d), when the mass of the object stayed the same while 
the slope angle varied, the relationship between the travelled distance and the time was obviously different. It can 
be further found that the larger the angle, the faster the sliding speed. Figure (e) shows that there are errors in us- 
ing different instruments to measure the experimental data, but it can be known that the travelled distance of the 
object is parabola to the time, and the travelled distance of the object gradually increased in unit time. Therefore, 
students thought that the slope motion was a constant acceleration. 


Figure 7 
The results of worksheet part of some students of the VL and MBL groups 
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A total of 118 questionnaires were filled out in this research, 60 for the VL group and 58 for the MBL group. 
Because of the different system functions of VL and MBL, the content of the questionnaires was somewhat different 
for the two groups. The Digital Platform Usability Questionnaire consists of four subscales, including Perceptual 
Usefulness (6 questions), Perceptual Ease of Use (2 questions), Use Attitude (3 questions), and Willingness to Use 
(2 questions). Using the Likert five-point scale, from 1 (very much disagree) to 5 (very much agree), the higher the 
answer score, the more the learner agrees that the digital learning system is usable. 

According to the analysis results of Table 3, the average of topics falls between “consent” and “ordinary’, 
indicating that most students thought that the virtual lab was usable and easy to use, and the attitude and the 
willingness of use were positive. It was known from the perceptual usefulness that this system was helpful for 
students to perform the actual experiments in the future. Because the system's capabilities made it easier for 
students to understand the relationship between experimental variables, this result means that the system was 


helpful for scientific modeling. 


Table 3 
Results of Virtual group questionnaire 


Questionnaire topics 
Subscale 1: Perceptual usefulness 
This learning system can help me learn the techniques and methods for experimentation. 


This learning system can improve my learning efficiency in hands-on experiments. 
This learning system can help me set up an experimental environment more easily. 


This learning system can help me set up an experimental environment faster. 


This learning system can help me understand the relationship of the experimental variables more easily. 


Overall, this learning system can help me better understand the physical experiment. 
Total score of subscale 1 

Subscale 2: Perceptual ease of use 

The functions provided by this learning system make it easy to do what | want to do. 
The features provided by this learning system are easy to use. 


Total score of subscale 2 
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ASS 


oT 


3.75 
3.86 
3.71 

3.86 
3.78 
22.73 


3.68 
3.75 
7.43 


SD 


id 


77 
8 
85 
68 
74 

4.64 


83 
86 
1.69 
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Questionnaire topics M SD 


Subscale 3: Use attitude 


| think this is a smart digital learning system. S03 14 
| think this is an attractive digital learning system. 3.41 85 
| think this is a pleasing digital learning system. 3.49 92 
Total score of sub scale 3 10.53 2.91 


Subscale 4: Willingness to use 


lf there is an opportunity, | hope to use this learning system frequently. 3.44 .90 
lf there is an opportunity, | would be happy to use this learning system. 3.61 81 
Total score of subscale 4 7.05 1.71 
Total score 47.74 10.55 


Table 4 shows the result of the usability questionnaire filled out by the MBL group. Most students thought that 
the MBL system was useful and easy to use, and the attitude and the willingness of use were positive. It was known 
from the perceptual usefulness that this system was useful for students to log and analyze experimental data. In 
the lab, students thought that the XY plot of the experimental data was helpful to understand the relationship 
between the experimental variables. Overall, these results mean that the system was helpful for scientific modeling. 


Table 4 
Results of MBL group questionnaire 


Questionnaire topics M SD 
Subscale 1: Perceptual usefulness 
This logging system can help me capture more detailed experimental data. 4.03 16 
This logging system can improve my ability to analyze experimental data. 4.02 .68 
This logging system can improve the efficiency of recording experimental data. 3.97 .80 
In this logging system, the function of plotting data into XY chart helps me understand the variable relation- 3.88 14 
ship in the experiment. 
This learning system can help me understand the variable relationship of the experiment more easily. 4.02 .68 
Overall, this learning system can help me better understand the actual experimentation. 3.93 3 
Total score of subscale 1 23.85 4.39 
Subscale 2: Perceptual ease of use 
The functions provided by this learning system make it easy to do what | want to do. 3.86 13 
The features provided by this learning system are easy to use. 3.86 84 
Total score of subscale 2 7.12 1.07 
Subscale 3: Use attitude 
| think this is a smart digital learning system. 3.81 .80 
| think this is an attractive digital learning system. 3.00 Oo 
| think this is a pleasing digital learning system. 3.63 .89 
Total score of subscale 3 11.02 202 
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Questionnaire topics M SD 


Subscale 4: Willingness to use 


lf there is an opportunity, | hope to use this learning system frequently. 3.66 .92 
lf there is an opportunity, | would be happy to use this learning system. 3.88 AZ 
Total score of subscale 4 7.94 1.64 
Total score 50.01 10.12 
Table 5 
Results of p-value from MBL group and VL group 
Questionnaire topics sda ' p-value 
mean score mean score 
Subscale 1: Perceptual usefulness 23.00 2245 0.124 
Subscale 2: Perceptual ease of use 7.73 7.43 0.256 
Subscale 3: Use attitude 11.02 10.54 0.262 
Subscale 4: Willingness to use 7.54 7.05 0.084 


For each subscale, the averaged sums of scores of between MBL group and VL group were compared. The 
results of t-test are shown in Table 5. All differences between the two groups were not significant. Nevertheless, all 
mean scores of MBL group were higher than those of VL group for all four subscales, indicating MBL were slightly 
preferred by students. 


C. The feedback of students 


After filling out the questionnaire, we asked students to answer an open-ended question voluntarily: “Can 
this system be improved? If so, please give suggestions.” 


1) Virtual Lab group 


| feel that the system helps increase my confidence and interest in running the physical slope experiment. 
After using the system, | understand the experimental process better and | want to do the physical 
experiment. 

| prefer to use a computer to run experiments and think that the computer data is more accurate and 
easy to analyze. 

| think that there will be errors in the physical experiment, so | want to do the physical experiment and 
understand the error relationship between the simulation and the physical experiment. 

| think the system helps me understand the slope movement experiment to a certain degree. 

This system helps me develop a better understanding of the experimental variable relationships. 

| suggest that the system can run more types of experiments. 


2) MBL group 


168 


Compared with the traditional use of a stopwatch to record data, | preferred to use a data logger to 
measure experimental data. 

| think the logging of data is convenient and accurate in MBL. 

| think this system helps me analyze experimental data better. 

| like this system, because MBL can get more accurate data in addition to learning the circuit. 
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e | think there is no human error in using MBL. 

e Because the stopwatch is very intuitive, | prefer using the stopwatch to record data. The data logger 
should be improved, | feel it was more troublesome than stopwatch. 

e |recommend using other sensors to improve accuracy, such as infrared. 


The VL group indicated the user friendliness of the simulation tool and the improvement of their under- 
standing of the meaning of the experiment, which was probably due to the simplification of the experimental 
procedure. The MBL group enjoyed the precision of the data logger, which facilitated data analysis. Some 
student of this group pointed out the stopwatch was more intuitive, probably due to the difficulty in using the 
data logger. Overall speaking, the students indicated the simulation tool and the data logging tool enriched 
their lab experiences. 


D. Pre-test and post-test 


The VL group and the MBL group took the same pre-test and post-test. Table 6 is the correctness rates of 
the pre-test and the post-test of the VL group and the MBL group and the p-value of the T-test. From the pre- 
test and post-test results, the VL group students’ correctness rate of answering the question “the effect of the 
mass on the acceleration of the sliding object” increased by 55%, “the effect of the mass on the acceleration of 
the sliding object” increased by 5% and “determine whether the sliding object moves at constant acceleration” 
increased by 18.33%. The MBL group students’ correctness rate of answering the question “the effect between 
the mass and the acceleration of the sliding object” increased by 14.75%, “the effect between the mass and 
the acceleration of the sliding object” increased by 1.59% and “determine whether the sliding object moves at 
constant acceleration” increased by 14.80%. The correctness rate of the post-test was higher than that of the 
pre-test. For Problem 1 and Problem 3, the improvement from the pre-test to the post-score was significant for 
both groups. For Problem 2, the improvement was not significant for both VL group and MBL group because 
their pre-test scores were as high as 91.67% and 98.41% respectively, which left little space for improvement. 

The pre-test scores of VL and those of MBL for the three problems (16.67% versus 14.28%, 91.67% versus 
98.41%, 80% versus 81.97% in Table 5) showed no noticeable differences, indicating that the prior knowledge of 
kinematics was similar for both groups. Regarding the post-test scores of both groups for the three problems, the 
correctness rates of the second and third problems were similar (96.67% versus 100%, 98.33% versus 96.77%). 
But the difference in post-test scores was obvious for the first problem: 71.6% versus 29.03%, meaning that the 
VL treatment was more effective than the MBL treatment. The difference was not due to the difference in prior 
knowledge since the pre-test scores (16.67% versus 14.28%) were similar. 


Table 6 
Pre-test and post-test correctness rate and p-value of VL group and MBL group 


VL MBL 
Problem 
Pre-test Post-test p-value Pre-test Post-test p-value 


1. Find the effect of the mass on the accelera- 
tion of the sliding object when the distance and 16.67% 71.67% <.0001 14.28% 29.03% 024 
angle of the slope are fixed. 


2. Find the effect of the slope angle on the 
acceleration of the sliding object when the 


0 0 0 0 
distance of the slope and mass of the object 91.67% 96.67% A18 98.41% 100% 321 
are fixed. 
3. Determine whether the sliding object moves 30.00% 98.33% een: sai, saan eee 


at constant acceleration. 
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Discussion 


For STEAM education, the ability to build and apply models for scientific inquiry is seen as an important part 
of the curriculum in engineering schools. Researchers have developed a virtual lab and an MBL system to help 
students conduct simulations and hands-on physics experiments to build scientific models. This study poses two 
research questions: 


1. How well does the web-based virtual lab system effectively help students do experiments and build 
scientific models? 
2. How well does the MBL system effectively help students do experiments and build scientific models? 


This research focuses on the use of models as a tool for inquiry, how students use the model to reason in the 
process of scientific inquiry, and repeatedly compare and revise their mental models (Pata & Sarapuu, 2006). Based 
onthe METT modeling teaching framework proposed by Schwarz and White (2005), an empirical research was car- 
ried out. From the students’ work sheets, they frequently varied the mass of the sliding object and the slope angles 
during the inquiry process and explored the influence of these variables on the outcome of travelled distance, and 
many of them finally built the correct model. The pretests showed that some students achieved a certain degree of 
understanding of the slope motion, but some students did not understand the impact of the controlled variables 
on the outcome. For example, for the activity “Exploring the relationship between the mass and acceleration of 
the sliding object when the distance and angle of the slope are fixed”, only 16.67% of the students in the VL group 
were correct, while only 14.28% of the MBL group was correct. For the activity “Exploring the relationship between 
slope angle and the acceleration of the sliding object when the distance of the slope and the mass of the object 
are fixed’, the two groups of students achieved correctness rates of more than 90%, indicating that they had prior 
knowledge on this topic. For the activity “Determine whether the slope motion of the object undergoes a constant 
acceleration’, the correctness rates of the two groups were about 80%, indicating that the remaining 20% students 
still did not understand the effect of object mass and slop angle on slope motion. 

From the pre-test results, it can be found that most students experienced much difficulty in “Exploring the 
relationship between the mass and the acceleration of the moving object when the travelling distance and the 
angle of the slope are fixed.” However, after using VL or MBL systems for experiments, the results of the work sheets 
and the posttest indicated that the two groups of students significantly improved their understanding, while the 
correctness rates for the other two activities also improved. 

It is also interesting to compare the performance of the two groups. Even though the posttest correctness 
rates of both groups improved significantly, the proportion of students in the VL group achieving correctness was 
much larger than that of the MBL group. In virtual lab, the simulation assumed zero friction between the moving 
object and the slope, students found that the mass of the object did not affect the outcome. However, since the 
experiments of the MBL group involved friction and measurement errors and the collected data did not match 
the textbook theory, the correctness rate of the posttest dropped. After the teacher explained the experiment at 
a conclusion session, the students should have a deeper understanding of the complications in MBL. 

After doing VL or MBL, the participants’ satisfaction on all topics in the questionnaire were favorable with 
average scores ranging from 3.41 to 4.03 (The best score would be 5). It indicated that most students thought both 
VL and MBL were useful and easy to use, and they were positive in their attitude and willing to use the systems. 
This result is consistent with the conclusions of Chou and Lu (2014). According to the feedback from the students, 
after the students used VL, they developed a better understanding, more confidence, and greater interest in the 
lab. For the MBL group, students considered the system to be easy to use and the collected data were accurate. 
However, there were also a small number of students who preferred to record data with a stopwatch, which was 
more intuitive, despite the fact that the collected data was much less than that collected with MBL. 

From the findings of Odeh, Shanab and Anabtawi (2015), it is known that hands-on lab and virtual lab each 
benefit students’ learning in different ways. Some researchers suggest that the current challenge is how to com- 
bine the two laboratories ina complementary way to achieve specific learning effects (Abdulwahed & Nagy, 2011; 
Heradio et al., 2016; Wiesner & Lan, 2004; Zacharia, 2007). In this empirical research, the results of the pre-test, the 
post-test and the responses of the questionnaire indicate that both VL and MBL were helpful for scientific modeling. 
However, MBL system took about three times longer to perform than VL did. If time and money are limited, then 
VL would be a better choice than MBL. But if hands-on techniques of handling the equipment are the target goals 
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of lab work, then MBL should be done instead. In addition, the students of the MBL group experienced problems 
in considering the friction between the moving object and the slope, as well as the measurement errors of the 
collected data. The results of the post-test indicated that the complications of friction and the measurement errors 
in MBL made the interpretation of the data more difficult for the students. To address this issue, the researchers 
proposed to arrange labs in four stages in future. First, VL is done with assumption of no friction. Second, VL is 
done with the consideration of friction in simulation. Third, VL is done with friction and an introduction of a small 
amount of random measurement errors. Finally, MBL is done. This strategy of adding complications step by step 
should be easier for the students to understand the authentic data of MBL in the final stage. If lab time is limited, 
then the second and third stages can be done with a demonstration by the instructor instead. 


Conclusions 


This research proposed two systems to assist students in scientific modeling, namely a virtual lab based on 
web technology and a low-cost MBL system and did an empirical experiment with two groups of first-year univer- 
sity students, namely the VL group and the MBL group. In VL, the students designed experiments with different 
values of several variables, ran the experiments, collected data and built a scientific model. The students could do 
VL with mobile devices with a web browser. The MBL system consists of a low-cost arduino expansion board with 
sensors, allowing students to learn the basic architecture of the microcontroller while building an experimental 
environment. In an era of emphasis on STEAM education, this system helps students learn the utility of modern 
electronics. The results of the questionnaire show that most students thought that VL and MBL were useful and easy 
to use, and they were positive in their attitude and expressed willingness to use the innovative technologies. From 
the results of the pretest and the posttest, students made progress in exploring the relation between the mass and 
the acceleration of the moving object down the slope when the distance and angle of the slope were fixed, as well 
as in determining whether the slope motion undergoes constant acceleration. In the Pre-test and Post-test, the 
p values of the Virtual group were < .01, and the MBL group were < .05 and <.01, respectively, indicating that the 
improvement of the scientific modeling by the two groups of students with the two systems were very significant. 
From the above preliminary results, it can be known that the VL and the MBL were helpful for students in scientific 
modeling. Last but not the least, considering the strengths and weaknesses of VL and MBL, some suggestions are 
made on how to combine the two approaches to provide more guidance to students in doing scientific modeling 
by introducing real-life complications in VL step by step before doing MBL as the final step. 
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